Diagnosability, Adequacy & Size: How Test Suites Impact Autograding

Automated grading is now prevalent in software engineering courses, typically assessing the correctness of students' programs using automated test suites. However, deficiencies in test suites could result in inconsistent grading. As such, we investigate how different test suites impact grades, and the extent to which their observable properties influence these grades. We build upon existing work, using students' solution programs, and test suites that we constructed using a sampling approach. We find that there is a high variation in grades from different test suites, with a standard deviation of ~10.1%. We further investigate how several properties of test suites influence these grades, including the number of tests, coverage, ability to detect other faults, and uniqueness. We use our findings to provide tutors with strategies for building test suites that evaluate students' software with consistency. These strategies include constructing test suites with high coverage, writing unique and diverse tests which evaluate solutions' correctness in different ways, and to run the tests against artificial faults to determine their quality

An empirical investigation into branch coverage for C programs using CUTE and AUSTIN

Automated test data generation has remained a topic of considerable interest for several decades because it lies at the heart of attempts to automate the process of Software Testing. This paper reports the results of an empirical study using the dynamic symbolic-execution tool. CUTE, and a search based tool, AUSTIN on five non-trivial open source applications. The aim is to provide practitioners with an assessment of what can be achieved by existing techniques with little or no specialist knowledge and to provide researchers with baseline data against which to measure subsequent work. To achieve this, each tool is applied 'as is', with neither additional tuning nor supporting harnesses and with no adjustments applied to the subject programs under test. The mere fact that these tools can be applied 'out of the box' in this manner reflects the growing maturity of Automated test data generation. However, as might be expected, the study reveals opportunities for improvement and suggests ways to hybridize these two approaches that have hitherto been developed entirely independently. (C) 2010 Elsevier Inc. All rights reserved

Design and analysis of different alternating variable searches for search-based software testing

Manual software testing is a notoriously expensive part of the software development process, and its automation is of high concern. One aspect of the testing process is the automatic generation of test inputs. This paper studies the Alternating Variable Method (AVM) approach to search-based test input generation. The AVM has been shown to be an effective and efficient means of generating branch-covering inputs for procedural programs. However, there has been little work that has sought to analyse the technique and further improve its performance. This paper proposes two different local searches that may be used in conjunction with the AVM, Geometric and Lattice Search. A theoretical runtime analysis proves that under certain conditions, the use of these searches results in better performance compared to the original AVM. These theoretical results are confirmed by an empirical study with five programs, which shows that increases of speed of over 50% are possible in practice

Symbolic search-based testing

We present an algorithm for constructing fitness functions that improve the efficiency of search-based testing when trying to generate branch adequate test data. The algorithm combines symbolic information with dynamic analysis and has two key advantages: It does not require any change in the underlying test data generation technique and it avoids many problems traditionally associated with symbolic execution, in particular the presence of loops. We have evaluated the algorithm on industrial closed source and open source systems using both local and global search-based testing techniques, demonstrating that both are statistically significantly more efficient using our approach. The test for significance was done using a one-sided, paired Wilcoxon signed rank test. On average, the local search requires 23.41% and the global search 7.78% fewer fitness evaluations when using a symbolic execution based fitness function generated by the algorithm

MTFuzz: Fuzzing with a Multi-Task Neural Network

Fuzzing is a widely used technique for detecting software bugs and vulnerabilities. Most popular fuzzers generate new inputs using an evolutionary search to maximize code coverage. Essentially, these fuzzers start with a set of seed inputs, mutate them to generate new inputs, and identify the promising inputs using an evolutionary fitness function for further mutation. Despite their success, evolutionary fuzzers tend to get stuck in long sequences of unproductive mutations. In recent years, machine learning (ML) based mutation strategies have reported promising results. However, the existing ML-based fuzzers are limited by the lack of quality and diversity of the training data. As the input space of the target programs is high dimensional and sparse, it is prohibitively expensive to collect many diverse samples demonstrating successful and unsuccessful mutations to train the model. In this paper, we address these issues by using a Multi-Task Neural Network that can learn a compact embedding of the input space based on diverse training samples for multiple related tasks (i.e., predicting for different types of coverage). The compact embedding can guide the mutation process by focusing most of the mutations on the parts of the embedding where the gradient is high. \tool uncovers $11$ previously unseen bugs and achieves an average of $2\times$ more edge coverage compared with 5 state-of-the-art fuzzer on 10 real-world programs.Comment: ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) 202

Agent based modelling helps in understanding the rules by which fibroblasts support keratinocyte colony formation

Background: Autologous keratincoytes are routinely expanded using irradiated mouse fibroblasts and bovine serum for clinical use. With growing concerns about the safety of these xenobiotic materials, it is desirable to culture keratinocytes in media without animal derived products. An improved understanding of epithelial/mesenchymal interactions could assist in this. Methodology/Principal Findings: A keratincyte/fibroblast o-culture model was developed by extending an agent-based keratinocyte colony formation model to include the response of keratinocytes to both fibroblasts and serum. The model was validated by comparison of the in virtuo and in vitro multicellular behaviour of keratinocytes and fibroblasts in single and co-culture in Greens medium. To test the robustness of the model, several properties of the fibroblasts were changed to investigate their influence on the multicellular morphogenesis of keratinocyes and fibroblasts. The model was then used to generate hypotheses to explore the interactions of both proliferative and growth arrested fibroblasts with keratinocytes. The key predictions arising from the model which were confirmed by in vitro experiments were that 1) the ratio of fibroblasts to keratinocytes would critically influence keratinocyte colony expansion, 2) this ratio needed to be optimum at the beginning of the co-culture, 3) proliferative fibroblasts would be more effective than irradiated cells in expanding keratinocytes and 4) in the presence of an adequate number of fibroblasts, keratinocyte expansion would be independent of serum. Conclusions: A closely associated computational and biological approach is a powerful tool for understanding complex biological systems such as the interactions between keratinocytes and fibroblasts. The key outcome of this study is the finding that the early addition of a critical ratio of proliferative fibroblasts can give rapid keratinocyte expansion without the use of irradiated mouse fibroblasts and bovine serum

Minimizing Errors in RT-PCR Detection and Quantification of SARS-CoV-2 RNA for Wastewater Surveillance

Wastewater surveillance for pathogens using the reverse transcription-polymerase chain reaction (RT-PCR) is an effective, resource-efficient tool for gathering additional community-level public health information, including the incidence and/or prevalence and trends of coronavirus disease-19 (COVID-19). Surveillance of SARS-CoV-2 in wastewater may provide an early-warning signal of COVID-19 infections in a community. The capacity of the world’s environmental microbiology and virology laboratories for SARS-CoV-2 RNA characterization in wastewater is rapidly increasing. However, there are no standardized protocols nor harmonized quality assurance and quality control (QA/QC) procedures for SARS-CoV-2 wastewater surveillance. This paper is a technical review of factors that can lead to false-positive and -negative errors in the surveillance of SARS-CoV-2, culminating in recommendations and strategies that can be implemented to identify and mitigate these errors. Recommendations include, stringent QA/QC measures, representative sampling approaches, effective virus concentration and efficient RNA extraction, amplification inhibition assessment, inclusion of sample processing controls, and considerations for RT-PCR assay selection and data interpretation. Clear data interpretation guidelines (e.g., determination of positive and negative samples) are critical, particularly during a low incidence of SARS-CoV-2 in wastewater. Corrective and confirmatory actions must be in place for inconclusive and/or potentially significant results (e.g., initial onset or reemergence of COVID-19 in a community). It will also be prudent to perform inter-laboratory comparisons to ensure results are reliable and interpretable for ongoing and retrospective analyses. The strategies that are recommended in this review aim to improve SARS-CoV-2 characterization for wastewater surveillance applications. A silver lining of the COVID-19 pandemic is that the efficacy of wastewater surveillance was demonstrated during this global crisis. In the future, wastewater will play an important role in the surveillance of a range of other communicable diseases.Highlights: Harmonized QA/QC procedures for SARS-CoV-2 wastewater surveillance are lacking; Wastewater analysis protocols are not optimized for trace analysis of viruses; False-positive and -negative errors have consequences for public health responses; Inter-laboratory studies utilizing standardized reference materials and protocols are needed.info:eu-repo/semantics/publishedVersio